Detection of viewing direction by microphone

ABSTRACT

A method for detecting of a viewing direction of a person includes sensing a speech signal originating from the person using at least one direction-selective microphone so as to produce a microphone signal, detecting the speaking direction of the person using the microphone signal, and deriving the viewing direction from the speaking direction. Also, a device that includes at least one direction-selective microphone configured to detect a speaking direction of the person using speech signals, and a processing unit configured to process the speech signals so as to determine the viewing direction of the person.

Priority is claimed to German Patent Application No. DE 102 17 822.4-35,filed on Apr. 17, 2002, which is incorporated by reference herein.

BACKGROUND

The present invention relates to a method and device for the detectionof a viewing direction of a person, in particular a driver of a vehicle.

In order to increase vehicle safety and vehicle comfort, modern motorvehicles increasingly have adaptive warning systems and informationsystems as well as backup systems which are capable of autonomouslyperforming functions in the field of vehicle guidance on the basis ofthe current traffic situation and the state of a driver of the vehicle.

For example, German Patent Document No. DE 197 34 307 C2 discloses adevice for monitoring and/or influencing the driving behaviour of adriver of a vehicle, which device generates a warning signal or acts onthe steering as a function of whether the driver's viewing directiondeviates from the direction of travel. The viewing direction of thedriver of the vehicle is determined here using a camera, with thedisadvantage that it is difficult to evaluate the camera signals inparticular in the dark or when the sun is low in the sky.

In order to be able to observe a driver of a vehicle by means of acamera system irrespective of the current lighting situation, inparticular at night, German Patent Application No. DE 101 60 843 A1,which was published at a later date than the present application,describes a lighting means for recording images in order to detectpatterns in motor vehicles, which illuminates the vehicle occupantsactively with infrared light. In order to avoid the driver of a vehiclebeing dazzled, it is not possible to use visible light for theillumination so that, in order to record images, it is also necessary touse a separate camera system which operates in the invisible wavelengthrange.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method and a device,suitable for carrying out the method, for the detection of the viewingdirection of persons, in particular of vehicle drivers, which canoperate independently of the currently prevailing lighting conditionsand without recourse to an active lighting device.

The present invention provides a method for the detection of a viewingdirection of a person, in particular a driver of a vehicle,characterized in that the viewing direction is derived from the speakingdirection of the speaking person, assuming that the speaking directioncorresponds essentially to the viewing direction, in which case, inorder to detect the speaking direction of the person, the speech signalsoriginating from said person are sensed by means of at least onedirection-selective microphone, and the speaking direction is determinedfrom the resulting microphone signal.

The present invention also provides a device for the detection of theviewing direction of a person, in particular of a driver of a vehicle,characterized in that the device comprises at least onedirection-selective microphone by means of which the speaking directionof the person can be detected by reference to speech signals, and inthat the device includes a unit for processing the speech signals andfor determining the viewing direction of the person.

Advantageous refinements and developments of the present invention aredescribed in the specification and the claims.

In the present invention, the speaking direction of a person is detectedin a particularly advantageous manner by at least onedirection-selective microphone, and the person's viewing direction isdetermined therefrom. The possibility of determining the viewingdirection of a speaker from the detected speaking direction is providedfor most situations which occur in reality as when a speaker is speakingto someone he also generally looks at them or at least tends to lookessentially in the direction in which the person is located. Inpractice, it is therefore generally perfectly possible to assume thatthe speaking direction essentially corresponds to the viewing direction.

BRIEF DESCRIPTION OF THE DRAWING

The present invention will be described below in detail by means ofexemplary embodiments and with reference to the drawing, in which:

FIG. 1 is a schematic view of a possible way of implementing the deviceaccording to the present invention for detecting the viewing directionin a motor vehicle.

DETAILED DESCRIPTION

The direction-selective microphone within the scope of the presentinvention may be either an individual microphone or a microphone arraywhich is formed from a multiplicity of individual microphones connectedto one another.

By way of example, reference will be made here to the German PatentDocument No. DE 38 43 034 C2 which discloses a microphone system,composed of a single microphone, for determining the direction of asound source, or to German Patent Document No. DE 34 47 842 A1 whichdescribes a microphone array for determining the direction of incidenceof sound waves, composed of a multiplicity of electro-acoustictransducers. Both German Patent Document No. DE 38 43 034 C2 and GermanPatent Document No. DE 34 47 842 A1, are incorporated by referenceherein.

An exemplary arrangement of the device according to the presentinvention in a motor vehicle for determining the viewing direction ofthe driver (2) of the vehicle is shown in the FIG. 1. Here, a driver (2)of a vehicle is seated on the driver's seat of a motor vehicle (1), inwhose interior three microphones (3 a, 3 b, 3 c) which interact as amicrophone array along the longitudinal axis of the motor vehicle pickup the sound waves (illustrated as arrows) which originate from thedriver. The microphone array is connected to a unit for processing thespeech signals (4), in which the signals of the microphone array areevaluated according to the present invention. If the driver is lookingin the travel direction (10) of the vehicle (straight ahead) as in theexample illustrated in the figure, the speech signal originating fromhim is received most strongly by the microphone (3 a) and most weakly bythe microphone (3 c). The signal intensity which is measured bymicrophone (3 b) should be only slightly less than the signal intensityof the signal measured at microphone (3 a) as although it is at a lessfavourable angle with respect to the vehicle driver (signal source) itis at a significantly smaller distance. In the situation, frequentlyoccurring when operating a vehicle, in which the vehicle driver turns toface his front seat passenger or an electronic device located in thecentre console of the vehicle, the ratio of the signal levels measuredat the microphones (3 a) and (3 b) with respect to one another changessignificantly in comparison with the situation in which the driver looksstraight ahead. The signal levels of the individual microphones (3 a–c)change even more significantly with respect to one another if the driverturns to a person located on the rear seat bench of the vehicle.

The arrangement of the microphone array illustrated in the figure ispurely exemplary, and of course quite different configurations, areconceivable, in particular depending on the field of application. Itwould, for example, also be conceivable to integrate more microphones inthe motor vehicle in the region along the dashboard in order to be ableto determine the viewing direction of the driver in the normal drivingmode even better while he views different areas of the surroundingsthrough the windscreen.

In enclosed spaces, in particular a motor vehicle, it is difficult todetermine the location of the speaker and detect the speaking directionowing to reflections of the speech signal at a wide variety ofreflective surfaces. It is therefore advantageously appropriate whendetecting the speech direction to take into account knowledge about thegeometry of the surroundings. It is particularly advantageous here ifknowledge about the conditions of the propagation of sound, especiallyrelating to the reflective properties of the objects surrounding thespeaker, can be taken into account. Such knowledge may be generated, onthe one hand, directly from the structural data of the surrounding areaand its known physical parameters, and on the other hand it is alsoconceivable to access specific standardization data which contains,inter alia, information relating to speech patterns of different personsand/or to different speaking directions. Within the scope of the methodaccording to the present invention it is then also advantageouslyconceivable to adapt such standardization data while the method isoperating. Such an adaptation can be carried out continuously or elsealso within the scope of an explicitly executed further standardization.For this purpose, within the scope of such a further standardization,speech patterns of different persons and/or speech patterns of at leastone person are stored for different speaking directions in accordancewith the procedure in a storage unit when the system is originallystandardized. It is advantageously possible, for the purpose of furthercalibration, to modify the storage method here in such a way that if aspeech pattern has already been stored in the storage unit for theinstantaneously speaking person and/or the instantaneous speakingdirection, said speech pattern is not replaced directly but rather onlymodified in accordance with a suitable algorithm. It is conceivable herefor the algorithm to calculate a weighted average from the speechpattern which has already been stored and the new speech pattern.

If the method for the detection of the viewing direction accessesstandardization data, the detection power can be significantly increasedif the standardization, or else the further standardization is carriedout according to fixed rules and specifications to the effect that whenthe speech patterns are generated, specific words or groups of words,for example, are used or that the speech patterns are generated byspeaking in predetermined, preferred speaking directions. In this way,the system can be selectively trained with respect to particularlycritical situations. Critical situations are, inter alia, speaking indirections in which, inter alia owing to the reflective characteristicsof the surroundings, the speaking direction can only be detected withdifficulty, or in which it has to be assumed, with certainty, that theperson to be monitored has turned his attention away from the regionwhich he is actually supposed to be observing to the extent that itresults in a high risk potential (for example in the situation in whicha driver of a vehicle turns to the persons on the rear seats and talksto them). In order to generate the standardization data it is of coursenot absolutely necessary for the respective speech patterns to begenerated by a natural person but rather it is equally conceivable forthe speech patterns to be generated synthetically. Within the frameworkof such a synthesis it is conceivable to connect speech patterns ofdifferent persons to one another or else to estimate such data on thebasis of theoretical considerations and with knowledge of the structuraldata of the surroundings of the direction-selective microphone. Use ofsuch synthetic standardization data is especially suitable for aworkshop and presetting of the detection system.

Within the framework of the detection of the speaking direction, it isalso particularly advantageously possible to use results of the speechevaluation of a communications system, for example of a telephonesystem, inter alia to reduce the production costs of the deviceaccording to the present invention. Communications systems which aresuitable for this are, for example, known from German Patent DocumentNo. DE 199 38 171 C2 which describes a system which selectively sensesacoustic signals from a plurality of predefined zones of the passengercompartment within a passenger car and passes them on to definedpositions/zones within the passenger compartment. In order to sense theacoustic signals, it is also proposed, inter alia, to use microphonearrays. It is then inventively conceivable to configure such acommunications system in such a way that, within the scope of theevaluation of the signals of the microphone arrays, the signalprocessing unit of the communications system supplies a detection resultwith respect to the speaking direction of a speaker, or else that atleast the signals of the microphone arrays are passed on to a separatedetection unit.

If the device according to the present invention is connected to backupsystems, it is possible, for example when monitoring a driver of avehicle, to interpret different distraction instances andinattentiveness, and thus output warning signals and information signalsof whatever kind to the driver of the vehicle in an adaptive and promptfashion, which signals considerably support said driver in his drivingfunctions.

It is of course also conceivable to operate the present invention incombination with other sensors (for example cameras or infrared contourdetection) in order to also improve the detection capacity further interms of acquiring more wide ranging knowledge about the personobserved. In this way, known systems for issuing warning signals andproviding information can be improved further.

The present invention is of course not only restricted to the use forthe detection of the viewing direction in a motor vehicle, but can beused to the same degree in any technical fields in which it is necessaryto detect and evaluate the viewing direction of persons.

1. Method for detecting a viewing direction of a person, the methodcomprising: sensing a speech signal originating from the person using atleast one microphone so as to produce a microphone signal; detecting thespeaking direction of the person using the microphone signal; andderiving the viewing direction from the speaking direction; wherein thespeech signal originates from a driver of a vehicle.
 2. The method asrecited in claim 1, wherein the sensing is performed using at least onedirection-selective individual microphone.
 3. The method as recited inclaim 1, wherein the sensing is performed using at least one array of aplurality of individual microphones connected to one another.
 4. Themethod as recited in claim 1, wherein the detection of the speakingdirection is performed using information related to a geometry of aspace surrounding the person and information related to a surroundingcondition for propagation of sound.
 5. The method as recited in claim 1,further comprising evaluating a speech of the person using acommunications system so as to obtain a result, and wherein thedetection of the speaking direction is performed using the result. 6.The method as recited in claim 1, further comprising providing firststandardization data including speech patterns relating to a pluralityof persons, and wherein the detection of the speaking direction isperformed using the first standardization data.
 7. The method as recitedin claim 1, further comprising providing second standardization datarelating to a plurality of different speaking directions, and whereinthe detection of the speaking direction is performed using the secondstandardization data.
 8. The method as recited in claim 6, furthercomprising, adapting the first standardization data during one of thesensing and detecting steps according to a changed situation.
 9. Themethod as recited in claim 8, wherein the changed situation includes oneof a changed way of speaking and a changed register.
 10. The method asrecited in claim 1, further comprising storing a speech pattern of atleast one person.
 11. The method as recited in claim 1, furthercomprising storing a speech pattern relating to a plurality of differentspeech directions using a storing device.
 12. The method as recited inclaim 10, further comprising inputting speech patterns of the personinto the storage device so as generate the speech pattern.
 13. Themethod as recited in claim 10, further comprising syntheticallygenerating data by combining speech patterns of a plurality of naturalpersons and inputting speech patterns of the synthetically generateddata.
 14. A device for detecting a viewing direction of a person,comprising: at least one microphone configured to detect a speakingdirection of the person using speech signals; and a processing unitconfigured to process the speech signals so as to determine the viewingdirection of the person; wherein the person is a driver of a vehicle.15. The device as recited in claim 14, wherein the at least onemicrophone includes a direction-selective individual microphone.
 16. Thedevice as recited in claim 14, wherein the at least one microphoneincludes a microphone array of a plurality of individual microphonesconnected to one another so as to provide direction-selectiveinformation.
 17. The device as recited in claim 14, further comprising astorage unit containing information related to a geometry influencingconditions of sound propagation and information related to a spacesurrounding the person being stored in the storage unit.
 18. The deviceas recited in claim 14, wherein further comprising a communicationssystem operatively connected to at least one of the microphone and theprocessing unit.
 19. The device as recited in claim 14, furthercomprising a storage unit containing standardization data relating tospeech patterns of a plurality of different persons.
 20. The device asrecited in claim 14, further comprising a storage unit containingstandardization data relating to a plurality of different speakingdirections.
 21. The device as recited in claim 19, further comprising anadaptation unit configured to adapt the standardization data accordingto a changed situation.
 22. The device as recited in claim 21, whereinthe changed situation includes a change in a way of speaking of theperson over time.
 23. The device as recited in claim 21, wherein thechanged situation includes a change in a register of the speaker overtime.
 24. The method as recited in claim 1, further comprisingevaluating a risk potential for the driver based on the viewingdirection.
 25. The device as recited in claim 14, wherein the processingunit is configured to evaluate a risk potential for the driver based onthe viewing direction.
 26. Method for detecting a viewing direction of avehicle driver, the method comprising: sensing a speech signaloriginating from the vehicle driver using at least one microphone so asto produce a microphone signal; detecting the speaking direction of thevehicle driver using the microphone signal; deriving the viewingdirection from the speaking direction; and evaluating a risk potentialfor the vehicle driver based on the viewing direction.